Paragraph retrieval for why-question answering Exploiting discourse structure for intelligent paragraph retrieval for why-QA

Finding answers to why-questions involves finding arguments in texts, rather than the noun phrases that are typical targets for factoid questions. Detecting arguments requires detecting specific rhetorical structures and relations. Therefore, we proposed the use of Rhetorical Structure Theory (RST) as a tool for discovering answer to why-questions in paragraphs that are likely to contain the answer. We evaluated this method using two sets of why-questions: one obtained by elicitation of native speakers and one containing questions that are asked to the online question answering system answers.com. Our procedure was able to find answers to about 60% of the why-questions. We conclude that some relation types have a high predictive power in answer selection, but we also found that many questions require a full paragraph for an answer. Therefore, we need to shift the research emphasis towards passage retrieval. We propose a three-step method for retrieving passages that are likely to contain the answer to a why-question: (1) query creation, (2) document retrieval and (3) paragraph retrieval and ranking. Standard information retrieval models are not suitable for ranking paragraphs as candidate answers. One issue is the small size of the text units that must be ranked. In addition, we need to incorporate information on the presence of RST relations in the language model used for ranking.